The more air that can pass through the core of the intercooler the greater the cooling efficiency of said intercooler. In some regard you are right in that you want to provide an exit for the air once it has gone through the intercooler as well.
A ducted intercooler can be almost 20% more efficient than if it were not ducted.